The present invention relates to the field of sub-pixel rendering techniques used on a matrix digital display. More specifically, the present invention relates to a method for improving the definition of black and white text and graphics on a color matrix digital display device.
Today, the state-of-the-art methods for improving the definition of black and white text graphics, including Microsoft ClearType technology, increase the potential display resolution of black and white text on a color matrix digital display device by using conventional sub-pixel rendering techniques. The improvement of the on-screen reading experience resulting from the sub-pixel rendering methods has enabled new product categories such as electronic books (eBooks). This improvement has also benefited the display of existing spreadsheets, word processing documents and Internet content which display text using fonts that have been rendered for color matrix displays. Font sub-pixel rendering is a well known technique, but it can not correct all problems with an image displayed on a color matrix digital display device. For example, sub-pixel rendering cannot apply to any text in which fonts have already been rendered, as in the case of a bitmapped image which contains text.
There are several types of sub-pixel rendering techniques in use today. One type is a sub-pixel horizontal resolution rendering used on digital matrix display devices. Another type is the horizontal definition improvement rendering used for digital text type. Another type of sub-pixel rendering is used for font design. These types are described in detail below.
A single pixel of a typical digital color matrix display device such as a liquid crystal device (LCD) display or a plasma display panel (PDP) display is composed of three in-line xe2x80x9csub-pixelsxe2x80x9d: one red, one green, and one blue (R-G-B). The sub-pixel triad forms a single pixel. The linear array of interleaved single-color sub-pixels translates to a horizontal resolution of three times the maximum horizontal resolution that could be achieved for the display. Therefore, addressing the actual sub-pixels individually and ignoring their differing colors could provide as much as three times the horizontal resolution from the existing digital matrix display panels than if single pixel addressing were used. Sub-pixel rendering works because human eyes perceive changes in luminance with greater resolution than changes in color.
Furthermore, when a white line is presented on a color matrix display, what really is being displayed is a line of sub-pixel triads of red, green and blue. The human eye does not perceive these closely spaced colors individually because the vision system does not see color changes at high resolution. Accordingly, the human eye mixes the three primary colors in combination to form intermediates. The eyes only register any of the three primary colors when single sub-pixels of the primary color signals are exclusively illuminated in an multi-pixel area. All other combinations of the primary color signals are perceived as intermediate (secondary and tertiary) color signals. The combination of all three color signals in the proper intensity is perceived as white and the absence of all color signals is perceived as black.
A Web Page on the Internet entitled xe2x80x9cSub-Pixel Rendering Technology, How it Worksxe2x80x9d at http://gre.com/ctwhat.htm, provides details and examples of sub-pixel rendering technology. One example from the Web Page is shown in FIG. 1a. This example shows that an object having a diagonal edge, 5 pixels wide, which is drawn based on a whole (RGB) white pixels creates a jagged edge. Similarly, FIG. 1b illustrates an example of a 5xc3x9715 pixel grid displaying a diagonal line travelling down 5 pixels. When the same objects are drawn based on sub-pixels, the diagonal edge becomes smooth, as shown in FIGS. 2a and 2b, respectively.
FIG. 3 illustrates that although the pixels along the edge are no longer white, they will be perceived as white by the eye because the sub-pixel color-bars are immediately adjacent to the other primary colors and the background is black. Therefore, by controlling the three sub-pixels of a color matrix display individually, greater horizontal resolution for (black and white) text or graphics can be achieved.
The conventional method for controlling the sub-pixels is through rendering. Rendering maps the pixels of the letters in a font onto the three sub-pixels in a particular sequence in order to achieve optimum resolution for the font. For example, FIG. 4a shows a 12 point regular (non-italics, non-bold) capital xe2x80x9cSxe2x80x9d rendered, using full-pixel rendering techniques, in Microsoft""s standard Times New Roman font. FIG. 4b illustrates what the capital xe2x80x9cSxe2x80x9d looks like at the sub-pixel level when the pixels shown in FIG. 4a are shifted one-third of a pixel to the right. The result is a blocky letter which may be difficult for the human eye to resolve. A technique known as xe2x80x98anti-aliasingxe2x80x99 was developed to make blocky letters easier to resolve. Using this technique, the image in FIG. 4c replaced the partially filled pixels each representing a proscribed gray level. A one-third filled pixel is assigned a light gray and a two-thirds filled pixel is assigned a dark gray. The eye will tend to average gray pixels with the adjacent pixels. FIG. 4d shows the anti-aliased letter rendered for a color matrix display, with the red-green-blue sequencing of the sub pixels elements. In this image, the coloration of the sub-pixels of the letter corresponds to the horizontal position of the visual energy.
Also described in the above-referenced Web Page are other areas that are suitable for sub-pixel rendering such as font character kerning. The process of font character kerning deals with managing the spacing between characters. Readability is enhanced by smoothing out the density of the resulting type. Kerning can be problematic at small point sizes due to the relatively large pixel size. When two characters are moved together by one pixel, they are too close. However, with sub-pixel rendering, character spacing can be tuned using a granularity of one-third of a pixel width for improved text readability.
Font embedding is another area suitable for sub-pixel rendering. Here the problem is bold face type. Sometimes xe2x80x9cboldxe2x80x9d text may seem too bold and at other times, it may not seem different from normal text. This is caused by the large relative horizontal width of display pixels. Typically, a letter is made xe2x80x9cboldxe2x80x9d by adding one or more horizontal pixels to the letter. Adding a pixel-width to a relatively small feature of a character may cause that feature to appear too bold while adding a pixel width to an already large feature may not appear to change the character at all. Sub-pixel rendering may be used to address at least part of this problem as the width of a character may be adjusted in units of one-third of a pixel.
Font Italicizing is another area suitable for sub-pixel rendering. Printed italic fonts can be problematic. Digital systems have encountered problems when attempting to render italics because the lines which form the letters all become slanted and, as described above, diagonal features do not translate well into large pixels. Here again, sub-pixel rendering allows the displaying of sloping features by fine-tuning the location and width of each pixel.
In general, it is a font designer who renders any given type (BandW) font with three times the horizontal resolution for LCD display applications. Therefore, a triple-width text image containing three times the rendering information (a three-times wider letter) is created, or the font is rendered at triple the horizontal width into a standard black and white region. However, the rendered text without additional processing would exhibit some undesirable artifact, known as color fringing. Color fringing occurs when adjacent rendered sub-pixels have the same color. For example, if the letter xe2x80x9cIxe2x80x9d is rendered in white-on-black text such that the first sub pixel in the stem is red and the last sub-pixel in the stem is green, then reading across the stem, the sub-pixel sequence may be RGBRG along the entire stem of the xe2x80x9cI.xe2x80x9d The vertically adjacent xe2x80x9cRGxe2x80x9d pixels at the right side of the stem may be seen as a yellow fringe on the side of the letter.
Sub-pixel rendering may result in color fringing because when any one or two of the color sub-pixels that make up a whole pixel unit are not simultaneously turned on or off, some color surrounding each character can be seen. For example, red is seen if a single red sub-pixel is on while magenta is seen if both red and blue sub-pixels are on. The remedy is to diffuse the extra color intensity through its immediate neighboring sub-pixels horizontally to vary the color of adjacent pixels in the rendered font. This, however, results in a blurring the letter similar to the anti-aliasing filtering of the letters as shown in FIGS. 4c and 4d. From the signal processing point of view, a lowpass FIR filter can be used to filter the text image horizontally in order to provide anti-aliasing filtering and to mitigate color fringing. The tradeoff between color-fringing artifact and image blurring can be made within the filter design such as bandwidth of the filter. This trade-off, however, usually produces an unsatisfactory result for small fonts. Either the character is too blurred or it exhibits noticeable color fringing.
Thus current sub-pixel rendering techniques have a major limitation. They cannot effectively be applied to a rendered font/text or graphics. In addition, these techniques require considerable effort by font designers and can not be applied to images that contain text such as bit-mapped images or compressed images (e.g. Tagged Image File Format (TIFF), Graphica Interchange Format (GIF) of Joint Pictures Experts Group (JPEG) images). Therefore, it is desirable to use additional digital signal processing techniques to expand the sub-pixel rendering methods beyond font/text creation. Specifically, there is a need for a post-process rendered BandW text or graphics image in typical bitmap form to achieve higher (horizontal) image resolution in a color matrix display.
The present invention is embodied in a digital signal processing (DSP) technique which finds and processes rendered text which is to be displayed on a color matrix display to increase its horizontal resolution. First, a black and white text/graphics (TG) detector is deployed to identify areas of the image which include TG. Next, the detected TG is processed with a morphological operation so that it approximates the fonts (or graphics) that would be generated from a sub-pixel rendering engine. Finally, the processed TG display data is filtered so as to minimize color fringing while maximizing its resolution. The resulting TG display data is then converted back to the sub-pixel (i.e., RGB) domain replacing the detected TG data in the image and the resulting composite image is displayed.